EDP 617: A Short Course on Evaluation Models

class: center, middle, inverse, title-slide

.title[
# EDP 617: A Short Course on Evaluation Models
]
.subtitle[
## A Tiny Primer
]
.author[
### Dr. Abhik Roy
]

---

section {
    height: 600px;
    width: 60%;
    margin: auto;
    border-radius: 21px;
    background-color: #212121;
}

.remark-slide-container {
background: #212121;
}

.hljs-github .hljs {
    background: transparent;
    color: #b2dfdb;
}

.hljs-github .hljs-keyword {
    color: #64b5f6;
}

.hljs-github .hljs-literal {
    color: #64b5f6;
}

.hljs-github .hljs-number {
    color: #64b5f6;
}

.hljs-github .hljs-string {
    color: #b7b3ef;
}

.hljs-github .hljs {
    background: transparent;
    color: #b2dfdb;
}

.hljs-github .hljs-keyword {
    color: #64b5f6;
}

.hljs-github .hljs-literal {
    color: #64b5f6;
}

.hljs-github .hljs-number {
    color: #64b5f6;
}

.hljs-github .hljs-string {
    color: #b7b3ef;
}

section p {
    text-align: center;
    font-size: 30px;
    background-color: #212121;
    border-radius: 21px;
    font-family: Roboto Condensed;
    font-style: bold;
    padding: 12px;
    color: #bff4ee;
    margin: auto;
}

#center {
text-align: center;
}

#right {
  text-align: right;
}

.center p {
  margin: 0;
  position: absolute;
  top: 50%;
  left: 50%;
  -ms-transform: translate(-50%, -50%);
  transform: translate(-50%, -50%);
}

.center2 {
  margin: 0;
  position: absolute;
  top: 50%;
  left: 50%;
  -ms-transform: translate(-50%, -50%);
  transform: translate(-50%, -50%);
}

.tab {
    display: inline-block;
    margin-left: 40px;
}

.tabdbl {
    display: inline-block;
    margin-left: 80px;
}

.tabtpl {
    display: inline-block;
    margin-left: 120px;
}

.obr
{
    display:block;
    margin-top:-15px;
}

.pull-left-left {
  float: left;
  width: 27%;
}

.pull-right-right {
  float: right;
  width: 32%;
}

img.expand:hover {
  margin: 0 auto;
  position: relative;
  width: 50%;
  display: flex;
  justify-content: center;
  align-items: center;
  align-content: center;
  transform: scale(1.5)             
  translateX(-35%);
  z-index: 99;
  transition:all 0.5s ease-in-out;
  -webkit-transition:all 0.2s ease-in-out;
}

.vertline {
  border-left: 5px solid #212121;
  height: 100px;
  margin-left: 15px;
  margin-right: 15px;
}

*, *:before, *:after {
	 box-sizing: border-box;
	 outline: none;
}

.hover {
	 position: relative;
	 display: flex;
	 align-items: center;
	 justify-content: center;
	 width: 400px;
	 height: 65px;
	 background-color: #e3c0ff;
	 border-radius: 99px;
	 box-shadow: 0 1px 3px rgba(0, 0, 0, 0.12), 0 1px 2px rgba(0, 0, 0, 0.24);
	 transition: all 0.3s cubic-bezier(0.25, 0.8, 0.25, 1);
	 overflow: hidden;
}

.hover:before, .hover:after {
	 position: absolute;
	 top: 0;
	 display: flex;
	 align-items: center;
   justify-content: center;
	 width: 50%;
	 height: 100%;
	 transition: 0.25s linear;
	 z-index: 1;
}

.hover:before {
	 content: '';
	 left: 0;
	 background-color: #ca86ec;
   color: #212121;
}

.hover:after {
	 content: '';
	 right: 0;
	 background-color: #d896ff;
}

.hover:hover {
	 background-color: #cc8bff;
	 box-shadow: 0 14px 28px rgba(0, 0, 0, 0.25), 0 10px 10px rgba(0, 0, 0, 0.22);
}

.hover:hover span {
	 opacity: 0;
	 z-index: -3;
}

.hover:hover:before {
	 opacity: 0.5;
	 transform: translateY(-100%);
}

.hover:hover:after {
	 opacity: 0.5;
	 transform: translateY(100%);
}

.hover span {
	 position: absolute;
	 top: 0;
	 left: 0;
	 display: flex;
	 align-items: center;
	 justify-content: center;
	 text-align: center;
	 width: 100%;
	 height: 100%;
	 color: #212121;
	 font-size: 24px;
	 font-weight: 700;
	 opacity: 1;
	 transition: opacity 0.25s;
	 z-index: 2;
   white-space:pre;
}

.hover .doc-link {
	 position: relative;
	 display: flex;
	 align-items: center;
	 justify-content: center;
	 text-align: center;
	 width: 25%;
	 height: 100%;
	 color: whitesmoke;
	 font-size: 24px;
	 text-decoration: none;
	 transition: 0.25s;
}
 .hover .doc-link i {
	 text-shadow: 1px 1px rgba(70, 98, 127, 0.7);
	 transform: scale(1);
}
 .hover .doc-link:hover {
	 background-color: rgba(245, 245, 245, 0.1);
}
 .hover .doc-link:hover i {
	 animation: bounce 0.4s linear;
}
 @keyframes bounce {
	 40% {
		 transform: scale(1.4);
	}
	 60% {
		 transform: scale(0.8);
	}
	 80% {
		 transform: scale(1.2);
	}
	 100% {
		 transform: scale(1);
	}
}

.boxl {
    width: 50%;
    margin: 5px;
    text-align: center;
}

.boxr {
    margin: 5px;
    text-align: center;
}

.picr {
    display: flex;
    justify-content: space-around;
    align-items: center;
}
</style>

<div>
<style type="text/css">.xaringan-extra-logo {
width: 110px;
height: 128px;
z-index: 0;
background-image: url(/Users/skynet/Documents/WVU/Teaching/GitHub.nosync/edp617/static/img/course_hex.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
top:1em;right:1em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('a')
          logo.classList = 'xaringan-extra-logo'
          logo.href = 'https://edp619.asocialdatascientist.com'
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

---

class: highlight-last-item
layout: true

---

#  Welcome

This is a relatively compact and short introduction to evaluation models.
<br>
<center>
<audio controls preload="auto">
  <source src="audio/models/S1 Introduction to EMD.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</center>

---

# The Basics

<center>
<audio controls preload="auto">
  <source src="audio/models/S2a The Basics.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</center>

<br>
<center>
<table class="table" style="width: auto !important; margin-left: auto; margin-right: auto;">
 
<tbody>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Behavioral Objectives Approach </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S2b%20Behavioral%20Objectives%20Approach.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> The Four-Level Model </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S2c%20The%20Four-Level%20Model.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Responsive Evaluation </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S2d%20The%20Four-Level%20Model.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Goal-Free Evaluation </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S2e%20Goal-Free%20Evaluation.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
</tbody>
</table>
</center>

---

# Middle of the Pack

<center>
<audio controls preload="auto">
  <source src="audio/models/S3a Middle Pack.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</center>

<br>
<center>
<table class="table" style="width: auto !important; margin-left: auto; margin-right: auto;">
 
<tbody>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Adversary/Judicial Approaches </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S3b%20Adversary-Judicial%20Approaches.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Consumer Oriented Approaches </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S3c%20Consumer%20Oriented%20Approaches.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Expertise/Accreditation Approaches </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S3d%20Expertise-Accreditation%20Approaches.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Organizational Learning </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S3e%20Organizational%20Learning.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; "> Success Case Method </td>
   <td style="text-align:center;width: 20em; "> <center>
<audio controls preload="auto">
  <source src="audio/models/S3f%20Success%20Case%20Method.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
</tbody>
</table>
</center>

---

# The Drivers

<center>
<audio controls preload="auto">
  <source src="audio/models/S4a Drivers.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</center>

<br>
<center>
<table class="table" style="width: auto !important; margin-left: auto; margin-right: auto;">
 
<tbody>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Theory Driven Evaluation </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S4b%20Theory%20Driven%20Evaluation.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Utilization Focused Evaluation </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S4c%20Utilization%20Focused%20Evaluation.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Participatory Evaluation </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S4d%20Participatory%20Evaluation.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
  <tr>
   <td style="text-align:center;width: 30em; vertical-align: middle !important; background: #212121"> Empowerment Evaluation </td>
   <td style="text-align:center;width: 20em; vertical-align: middle !important; background: #212121"> <center>
<audio controls preload="auto">
  <source src="audio/models/S4e%20Empowerment%20Evaluation.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</audio></source></audio>
</center> </td>
  </tr>
</tbody>
</table>
</center>

---

# Which One to Use?

You don't have to formally declare a model. In fact, unless specific circumstances demand it, its better that you probably don't. Recently there has been a push to get away from a design oriented evaluation approach which is good for many reasons - for one you aren't pigeonholing yourself to a predetermined structure that

1. may not apply exactly resulting in skewed results

2. dilutes or suppresses creativity and thinking

This allows you to deal with the program as is rather than viewing it through framework of a model. The bottom line is if we believe that *no two programs are alike, then why should we use a one size fit all approach to study it?*

---

# So should you care?

Yes because even though a prior framework is likely not the best tool to use, you'll use parts of multiple models here and there. So rather than treating these as musts, think of them as possible approaches that can be combined, expanded, ignored, etc. Like anything else, they can help but most evaluations should never rely on any one of them.
<br>
<center>
<audio controls preload="auto">
  <source src="audio/models/S5 Which One.mp3" type="audio/mpeg">
  Your browser does not support embedded audio.
<audio>
</center>

---

# And That's It

Remember that models don't determine the course of an evaluation - you do! Use your best judgment, common sense, content knowledge, expertise, etc - the things that hopefully distinguish you both professionally and academically to administer a study. If you have any questions, please feel free to reach out.

<center>
<br><br>
<div class="fade_rule"></div>  
<br><br>
</center>
<br><br><br><br><br><br><br><br>
<center>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><br />This work is licensed under a <br /><a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>
</center>